Selective use of adaptive in-loop color-space transform and other video coding tools

ABSTRACT

A method of video processing is provided. The method includes: determining, for a conversion between a current video unit of a video and a coded representation of the video, that applicability of a first coding tool and a second coding tool is mutually exclusive; and performing the conversion based on the determining, wherein the first coding tool corresponds to an adaptive color space transformation (ACT) tool; wherein use of the ACT tool comprises: converting, during encoding a representation of a visual signal from a first color domain to a second color domain, or converting during decoding, a representation of a visual signal from the second color domain to the first color domain.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2020/097370, filed on Jun. 22, 2020, which claims the priorityto and benefits of International Patent Application PCT/CN2019/092326,filed on Jun. 21, 2019. All the aforementioned patent applications arehereby incorporated by reference in their entireties.

TECHNICAL FIELD

This patent document is directed generally to video coding and decodingtechnologies.

BACKGROUND

Video coding standards have evolved primarily through the development ofthe well-known ITU-T and ISO/IEC standards. The ITU-T produced H.261 andH.263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the twoorganizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4Advanced Video Coding (AVC) and H.265/High Efficiency Video Coding(HEVC) standards. Since H.262, the video coding standards are based onthe hybrid video coding structure wherein temporal prediction plustransform coding are utilized. To explore the future video codingtechnologies beyond HEVC, Joint Video Exploration Team (JVET) wasfounded by VCEG and MPEG jointly in 2015. Since then, many new methodshave been adopted by JVET and put into the reference software namedJoint Exploration Model (JEM). In April 2018, the Joint Video ExpertTeam (JVET) between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) wascreated to work on the next generation Versatile Video Coding (VVC)standard targeting at 50% bitrate reduction compared to HEVC.

SUMMARY

Using the disclosed video coding, transcoding or decoding techniques,embodiments of video encoders or decoders can handle virtual boundariesof coding tree blocks to provide better compression efficiency andsimpler implementations of coding or decoding tools.

In one example aspect, a method of video processing is disclosed. Themethod includes determining, due to a dual tree partitioning structurebeing used for a conversion between a video unit and a codedrepresentation of the video unit, that use of an adaptive color spacetransformation (ACT) tool is disabled for the video unit; andperforming, based on the determining, the conversion by disabling theACT tool for the video unit, wherein the use of the ACT tool comprises:converting, during encoding, a representation of a visual signal from afirst color domain to a second color domain, or converting, duringdecoding, a representation of a visual signal from the second colordomain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes determining that a dual tree partitioningstructure and an adaptive color space transformation (ACT) tool are usedfor a conversion between a video unit and a coded representation of thevideo unit; and performing, based on the determining, the conversion byenabling the ACT tool for the video unit, wherein use of the ACT toolcomprises: converting during encoding, a representation of a visualsignal from a first color domain to a second color domain, orconverting, during decoding, a representation of a visual signal fromthe second color domain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes determining, for a conversion between acurrent video block of a video and a coded representation of the video,that applicability of a first coding tool and a second coding tool ismutually exclusive; and performing the conversion based on thedetermining, wherein the first coding tool corresponds to an adaptivecolor space transformation (ACT) tool; wherein use of the ACT toolcomprises: converting, during encoding, a representation of a visualsignal from a first color domain to a second color domain, orconverting, during decoding, a representation of a visual signal fromthe second color domain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes determining that both a coding tool andan adaptive color space transformation (ACT) tool are used for aconversion between a current video block of a video and a codedrepresentation of the video; and performing, based on the determining,the conversion by enabling the ACT tool for the current video block,wherein use of the ACT tool comprises: converting, during encoding arepresentation of a visual signal from a first color domain to a secondcolor domain, or converting, during decoding, a representation of avisual signal from the second color domain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes determining, for a conversion between acurrent video block of a video unit of a video and a codedrepresentation of the video, that an adaptive color space transformation(ACT) tool is disabled for the conversion due to an in-loop reshaping(ILR) tool being enabled for the video unit; and performing, based onthe determining, the conversion, and wherein the use of the ILR toolincludes constructing the video unit based on a luma reshaping between afirst domain and a second domain and/or a chroma residue scaling in aluma-dependent manner, and wherein use of the ACT tool comprises:converting, during encoding, a representation of a visual signal from afirst color domain to a second color domain, or converting, duringdecoding, a representation of a visual signal from the second colordomain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes determining that both an in-loopreshaping (ILR) tool and an adaptive color space transformation (ACT)tool are enabled for a conversion between a video unit and a codedrepresentation of the video unit; and performing, based on thedetermining, the conversion, and wherein use of the ILR tool includesconstructing the current video unit based on a first domain and a seconddomain and/or scaling chroma residue in a luma-dependent manner, andwherein use of the ACT tool comprises: converting, during encoding, arepresentation of a visual signal from a first color domain to a secondcolor domain, or converting, during decoding, a representation of avisual signal from the second color domain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes determining that both a sub-blocktransform (SBT) tool and an adaptive color space transformation (ACT)coding tool are enabled fora conversion between a current video blockand a coded representation of the current video block; and performing,based on the determining, the conversion, wherein use of the SBT toolcomprises applying a transform process or an inverse transform processon a sub-part of a prediction residual block, and wherein use of the ACTtool comprises: converting, during encoding, a representation of avisual signal from a first color domain to a second color domain, orconverting, during decoding, a representation of a visual signal fromthe second color domain to the first color domain.

In another example aspect, another method of video processing isdisclosed. The method includes performing a conversion between a videounit of a video and a coded representation of the video, where the videounit comprises one or more partitions at a first level comprising one ormore partitions at a second level, wherein the coded representationconforms to a formatting rule, wherein the formatting rule specifieswhether to include, or a partition level at which a syntax elementindicative of use of an adaptive color space transformation (ACT) toolfor representing the one or more second level partitions in the codedrepresentation is included in the coded representation, wherein thepartition level is one of the first level, the second level or the videounit.

In yet another example aspect, a video encoding apparatus configured toperform an above-described method is disclosed.

In yet another example aspect, a video decoder that is configured toperform an above-described method is disclosed.

In yet another example aspect, a machine-readable medium is disclosed.The medium stores code which, upon execution, causes a processor toimplement one or more of the above-described methods.

The above and other aspects and features of the disclosed technology aredescribed in greater detail in the drawings, the description and theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of encoding flow with adaptive color-spacetransform (ACT).

FIG. 2 shows an example of a decoding flow with ACT.

FIG. 3 shows an example of neighbouring samples used for deriving ICparameters.

FIG. 4 shows an example flowchart of decoding flow with reshaping.

FIG. 5 is a reference line example.

FIG. 6 shows example of division of 4×8 and 8×4 blocks.

FIG. 7 shows an example of division of all blocks except 4×8, 8×4 and4×4.

FIG. 8 is an illustration of ALWIP for 4×4 blocks.

FIG. 9 is an illustration of ALWIP for 8×8 blocks.

FIG. 10 is an illustration of ALWIP for 8×4 blocks.

FIG. 11 is an illustration of ALWIP for 16×16 blocks.

FIG. 12 is an illustration of sub-block transform modes SBT-V and SBT-H(The grey area is a TU which may have non-zero coefficients; the whitearea is a zero-out TU with all zero coefficients).

FIG. 13 is an illustration of sub-block transform modes SBT-Q.

FIGS. 14 and 15 are block diagrams of an example apparatus for videoprocessing.

FIGS. 16A and 16B are flowcharts for example methods of video processingbased on some implementations of the disclosed technology.

FIGS. 17A to 17E are flowcharts for example methods of video processingbased on some implementations of the disclosed technology.

FIG. 18 is a flowchart for an example method of video processing basedon some implementations of the disclosed technology.

DETAILED DESCRIPTION

Section headings are used in the present document to facilitate ease ofunderstanding and do not limit the embodiments disclosed in a section toonly that section. Furthermore, while certain embodiments are describedwith reference to Versatile Video Coding or other specific video codecs,the disclosed techniques are applicable to other video codingtechnologies also. Furthermore, while some embodiments describe videocoding steps in detail, it will be understood that corresponding stepsdecoding that undo the coding will be implemented by a decoder.Furthermore, the term video processing encompasses video coding orcompression, video decoding or decompression and video transcoding inwhich video pixels are represented from one compressed format intoanother compressed format or at a different compressed bitrate.

1. SUMMARY

This document is related to video coding technologies. Specifically, itis related to interactions of adaptive color-space transform with othertools in video coding. It may be applied to the existing video codingstandard like HEVC, or the standard (Versatile Video Coding) to befinalized. It may be also applicable to future video coding standards orvideo codec.

2. BACKGROUND

Video coding standards have evolved primarily through the development ofthe well-known ITU-T and ISO/IEC standards. The ITU-T produced H.261 andH.263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the twoorganizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4Advanced Video Coding (AVC) and H.265/HEVC standards. Since H.262, thevideo coding standards are based on the hybrid video coding structurewherein temporal prediction plus transform coding are utilized. Toexplore the future video coding technologies beyond HEVC, Joint VideoExploration Team (JVET) was founded by VCEG and MPEG jointly in 2015.Since then, many new methods have been adopted by JVET and put into thereference software named Joint Exploration Model (JEM). In April 2018,the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1SC29/WG11 (MPEG) was created to work on the VVC standard targeting at50% bitrate reduction compared to HEVC.

The latest version of VVC draft, i.e., Versatile Video Coding (Draft 5)could be found at:

http://phenix.it-sudparis.eu/jvet/doc_end_user/documents/14_Geneva/wg11/JVET-N1001-v2.zip

The latest reference software of VVC, named VTM, could be found at:

https://vcgit.hhi.fraunhofer.de/jvetNVCSoftware_VTM/tags/VTM-5.0

2.1. Adaptive Color-Space Transform (ACT) in HEVC Screen Content CodingExtensions

In the HEVC SCC Extensions, several tools have been proposed andemployed to improve the SCC efficiency under the HEVC framework. Forexample, to exploit the repeated patterns in SC, an intra block copying(IBC) scheme was adopted. Similar to the motion compensation scheme usedfor inter pictures, the IBC mode searches for the repeated patterns inthe already reconstructed region of the current picture. Anotherdirection to improve SCC is to reduce the inter-color-componentredundancy for the RGB/YCbCr sequences in the 4:4:4 chroma format.

The cross-component prediction (CCP) technology signals a weightingparameter index for each chroma color component of a transform unit. CCPprovides good coding efficiency improvements with limited addedcomplexity and thus, it was adopted to the HEVC Range Extensions and ispart of HEVC Ver. 2 which specifies descriptions of Range Extensions,and other Extensions.

In order to further exploit inter-color-components correlation for SCC,an in-loop adaptive color-space transform (ACT) for HEVC SCC Extensionswas employed. The basic idea of ACT is to adaptively convert theprediction residual into a color space with reduced redundancy among thethree-color components. Before and after that, the signal follows theexisting coding path in HEVC Range Extensions. To keep the complexity aslow as possible, only one additional color-space (i.e., RGB to YCgCo-Rinverse transform) is considered, which can be easily implemented withshift and add operations.

2.1.1. Transforms Used in ACT

For lossy coding, the YCgCo transform is used while its reversiblevariant, i.e., YCgCo-R, is used for lossless coding.

The forward and inverse YCgCo transform process is listed as below:taking a pixel in (R, G, B) color format as an example:

${{Forwar}{\text{d:}\mspace{14mu}\begin{bmatrix}Y \\{Cg} \\{Co}\end{bmatrix}}} = {\frac{1}{4}*{\begin{bmatrix}1 & 2 & 1 \\{- 1} & 2 & {- 1} \\2 & 0 & {- 2}\end{bmatrix}\begin{bmatrix}R \\G \\B\end{bmatrix}}}$ ${{Invers}{\text{e:}\mspace{14mu}\begin{bmatrix}R \\G \\B\end{bmatrix}}} = {\begin{bmatrix}1 & {- 2} & 1 \\1 & 1 & 0 \\1 & {- 1} & {- 1}\end{bmatrix}\begin{bmatrix}Y \\{Cg} \\{Co}\end{bmatrix}}$

Different from YCgCo transform which could be implemented by a matrixmultiplication, the reversible color-space transform, i.e., YCgCo-R,used in ACT can only be performed in lifting-based operation as follows:

${Forwar}\text{d:}\mspace{14mu}\begin{matrix}{{Co} = {R - B}} \\{t = {B + \left( {{Co} ⪢ 1} \right)}} \\{{Cg} = {G - t}} \\{Y = {t + \left( {{Cg} ⪢ 1} \right)}}\end{matrix}$ ${Invers}\text{e:}\mspace{14mu}\begin{matrix}{t = {Y - \left( {{Cg} ⪢ 1} \right)}} \\{G = {{Cg} + t}} \\{B = {t - \left( {{Co} ⪢ 1} \right)}} \\{R = {{Co} + b}}\end{matrix}$

2.1.2. Usage of ACT

For each TU, a flag may be signaled to indicate the usage of color-spacetransform. In addition, for intra coded CUs, ACT is enabled only whenthe chroma and luma intra prediction modes are the same, i.e., thechroma block is coded with DM mode.

FIG. 1 shows the block diagram of the proposed method at the encoderwith the residual signal derived from intra/inter prediction as theinput. The proposed function blocks, including forward and reversecolor-space transforms, are located in the coding loop and highlighted.As shown in FIG. 1 , after the intra- or inter-prediction process(including the prediction process for IBC mode), it is determinedwhether to perform the forward color-space transform. With theintroduced color-space transform, the color space of the input signalmay be converted to YCgCo with less correlation among the three-colorcomponents. After that, the original coding flow, such as CCP, integertransform (i.e., T in FIG. 1 ), if applicable, quantization (i.e., Q inFIG. 1 ) and entropy coding processes, is further invoked in order.Meanwhile, during the reconstruction or decoding process as depicted inFIG. 2 , after the conventional inverse quantization (i.e., IQ in FIG. 2), inverse transform (i.e., IT in FIG. 2 ) and inverse CCP, ifapplicable, the inverse color transform is invoked to convert the codedresidual back to the original color space. It should be noted that, thecolor-space conversion process is applied to the residual signal insteadof the reconstruction signal. With such a method, the decoder only needsto perform the inverse color space transform process which could keepthe complexity increase as low as possible. Furthermore, in ACT, fixedcolor space transforms, i.e., YCgCo and YCgCo-R, are utilized regardlessof input color spaces.

2.2. Local Illumination Compensation in JEM

Local Illumination Compensation (LIC) is based on a linear model forillumination changes, using a scaling factor a and an offset b. And itis enabled or disabled adaptively for each inter-mode coded coding unit(CU).

When LIC applies for a CU, a least square error method is employed toderive the parameters a and b by using the neighbouring samples of thecurrent CU and their corresponding reference samples. More specifically,as illustrated in FIG. 3 , the sub sampled (2:1 sub sampling)neighbouring samples of the CU and the corresponding samples (identifiedby motion information of the current CU or sub-CU) in the referencepicture are used.

2.2.1. Derivation of Prediction Blocks

The IC parameters are derived and applied for each prediction directionseparately. For each prediction direction, a first prediction block isgenerated with the decoded motion information, then a temporaryprediction block is obtained via applying the LIC model. Afterwards, thetwo temporary prediction blocks are utilized to derive the finalprediction block.

When a CU is coded with merge mode, the LIC flag is copied fromneighbouring blocks, in a way similar to motion information copy inmerge mode; otherwise, an LIC flag is signalled for the CU to indicatewhether LIC applies or not.

When LIC is enabled for a picture, additional CU level RD check isneeded to determine whether LIC is applied or not for a CU. When LIC isenabled for a CU, mean-removed sum of absolute difference (MR-SAD) andmean-removed sum of absolute Hadamard-transformed difference (MR-SATD)are used, instead of SAD and SATD, for integer pel motion search andfractional pel motion search, respectively.

To reduce the encoding complexity, the following encoding scheme isapplied in the JEM.

-   -   LIC is disabled for the entire picture when there is no obvious        illumination change between a current picture and its reference        pictures. To identify this situation, histograms of a current        picture and every reference picture of the current picture are        calculated at the encoder. If the histogram difference between        the current picture and every reference picture of the current        picture is smaller than a given threshold, LIC is disabled for        the current picture; otherwise, LIC is enabled for the current        picture.

2.3. Inter Prediction Methods in VVC

There are several new coding tools for inter prediction improvement,such as Adaptive motion vector difference resolution (AMVR) forsignaling MVD, affine prediction mode, Triangular prediction mode (TPM),ATMVP, Generalized Bi-Prediction (GBI), Bi-directional Optical flow(BIO).

2.3.1. Coding Block Structure in VVC

In VVC, a Quad-Tree/Binary-Tree/Ternary Tree (QT/BT/TT) structure isadopted to divide a picture into square or rectangle blocks.

Besides QT/BT/TT, separate tree (a.k.a. Dual coding tree) is alsoadopted in VVC for I-frames. With separate tree, the coding blockstructure are signaled separately for the luma and chroma components.

2.4. In-Loop Reshaping (ILR) in JVET-M0427

The basic idea of in-loop reshaping (ILR) is to convert the original (inthe first domain) signal (prediction/reconstruction signal) to a seconddomain (reshaped domain).

The in-loop luma reshaper is implemented as a pair of look-up tables(LUTs), but only one of the two LUTs need to be signaled as the otherone can be computed from the signaled LUT. Each LUT is aone-dimensional, 10-bit, 1024-entry mapping table (1D-LUT). One LUT is aforward LUT, FwdLUT, that maps input luma code values Y_(i) to alteredvalues Y_(r): Y_(r)=FwdLUT [Y_(i)]. The other LUT is an inverse LUT,InvLUT, that maps altered code values Y_(r) to Ŷ_(i):Ŷ_(i)=InvLUT[Y_(r)]. (Ŷ_(i) represents the reconstruction values ofY_(i)).

2.4.1. PWL Model

Conceptually, piece-wise linear (PWL) is implemented in the followingway:

Let x1, x2 be two input pivot points, and y1, y2 be their correspondingoutput pivot points for one piece. The output value y for any inputvalue x between x1 and x2 can be interpolated by the following equation:y=((y2−y1)/(x2−x1))*(x−x1)+y1

In fixed point implementation, the equation can be rewritten as:y=((m*x+2^(FP_PREC-1))>>FP_PREC)+c

where m is scalar, c is an offset, and FP_PREC is a constant value tospecify the precision.

Note that in CE-12 software, the PWL model is used to precompute the1024-entry FwdLUT and InvLUT mapping tables; but the PWL model alsoallows implementations to calculate identical mapping values on-the-flywithout pre-computing the LUTs.

2.4.2. Luma Reshaping

Test 2 of the in-loop luma reshaping (i.e., CE12-2 in the proposal)provides a lower complexity pipeline that also eliminates decodinglatency for block-wise intra prediction in inter slice reconstruction.Intra prediction is performed in reshaped domain for both inter andintra slices.

Intra prediction is always performed in reshaped domain regardless ofslice type. With such arrangement, intra prediction can startimmediately after previous TU reconstruction is done. Such arrangementcan also provide a unified process for intra mode instead of being slicedependent FIG. 4 shows the block diagram of the CE12-2 decoding processbased on mode.

CE12-2 also tests 16-piece piece-wise linear (PWL) models for luma andchroma residue scaling instead of the 32-piece PWL models of CE12-1.

FIG. 4 shows a flowchart of decoding flow with reshaping.

Inter slice reconstruction with in-loop luma reshaper in CE12-2(light-green shaded blocks indicate signal in reshaped domain: lumaresidue; intra luma predicted; and intra luma reconstructed)

2.4.3. Luma-Dependent Chroma Residue Scaling

Luma-dependent chroma residue scaling is a multiplicative processimplemented with fixed-point integer operation. Chroma residue scalingcompensates for luma signal interaction with the chroma signal. Chromaresidue scaling is applied at the TU level. More specifically, thefollowing applies:

-   -   For intra, the reconstructed luma is averaged.    -   For inter, the prediction luma is averaged.

The average is used to identify an index in a PWL model. The indexidentifies a scaling factor cScaleInv. The chroma residual is multipliedby that number.

It is noted that the chroma scaling factor is calculated fromforward-mapped predicted luma values rather than reconstructed lumavalues

2.4.3.1. Signalling of ILR Side Information

The parameters are (currently) sent in the tile group header (similar toALF). These reportedly take 40-100 bits.

2.4.3.2. Usage of ILR

At the encoder side, each picture (or tile group) is firstly convertedto the reshaped domain. And all the coding process is performed in thereshaped domain. For intra prediction, the neighboring block is in thereshaped domain; for inter prediction, the reference blocks (generatedfrom the original domain from decoded picture buffer) are firstlyconverted to the reshaped domain. Then the residual is generated andcoded to the bitstream.

After the whole picture (or tile group) finishes encoding/decoding,samples in the reshaped domain are converted to the original domain,then deblocking filter and other filters are applied.

Forward reshaping to the prediction signal is disabled for the followingcases:

-   -   Current Block is Intra-Coded    -   Current block is coded as CPR (current picture referencing, aka        intra block copy, IBC)    -   Current block is coded as combined inter-intra mode (CIIP) and        the forward reshaping is disabled for the intra prediction block

2.5. Virtual Pipelining Data Units (VPDU)

Virtual pipeline data units (VPDUs) are defined as non-overlappingM×M-luma (L)/N×N-chroma(C) units in a picture. In hardware decoders,successive VPDUs are processed by multiple pipeline stages at the sametime; different stages process different VPDUs simultaneously. The VPDUsize is roughly proportional to the buffer size in most pipeline stages,so it is said to be very important to keep the VPDU size small. In HEVChardware decoders, the VPDU size is set to the maximum transform block(TB) size. Enlarging the maximum TB size from 32×32-L/16×16-C (as inHEVC) to 64×64-L/32×32-C (as in the current VVC) can bring coding gains,which results in 4× of VPDU size (64×64-L/32×32-C) expectedly incomparison with HEVC. However, in addition to quadtree (QT) coding unit(CU) partitioning, ternary tree (TT) and binary tree (BT) are adopted inVVC for achieving additional coding gains, and TT and BT splits can beapplied to 128×128-L/64×64-C coding tree blocks (CTUs) recursively,which is said to lead to 16× of VPDU size (128×128-L/64×64-C) incomparison with HEVC.

In current design of VVC, the VPDU size is defined as 64×64-L/32×32-C.

2.6. Multiple Reference Line (MRL)

Multiple reference line (MRL) intra prediction uses more reference linesfor intra prediction. In FIG. 5 , an example of 4 reference lines isdepicted, where the samples of segments A and F are not fetched fromreconstructed neighbouring samples but padded with the closest samplesfrom Segment B and E, respectively. HEVC intra-picture prediction usesthe nearest reference line (i.e., reference line 0). In MRL, 2additional lines (reference line 1 and reference line 3) are used.

The index of selected reference line (mrl_idx) is signaled and used togenerate intra predictor. For reference line index, which is greaterthan 0, only include additional reference line modes in MPM list andonly signal MPM index without remaining mode. The reference line indexis signaled before intra prediction modes, and Planar and DC modes areexcluded from intra prediction modes in case a nonzero reference lineindex is signaled.

FIG. 5 is an example of four reference lines neighboring to a predictionblock.

MRL is disabled for the first line of blocks inside a CTU to preventusing extended reference samples outside the current CTU line. Also,PDPC is disabled when additional line is used.

2.7. Intra Subblock Partitioning (ISP)

In JVET-M0102, ISP is proposed, which divides luma intra-predictedblocks vertically or horizontally into 2 or 4 sub-partitions dependingon the block size dimensions, as shown in Table 1. FIG. 6 and FIG. 7show examples of the two possibilities. All sub-partitions fulfill thecondition of having at least 16 samples. For block sizes, 4×N or N×4(with N>8), if allowed, the 1×N or N×1 sub-partition may exist.

TABLE 1 Number of sub-partitions depending on the block size (denotedmaximum transform size by maxTB Size) Splitting Number of Sub- directionBlock Size Partitions N/A minimum transform size Not divided 4 × 8:horizontal 4 × 8 and 8 × 4 2 8 × 4: vertical Signaled If neither 4 × 8nor 8 × 4, and W <= maxTBSize and 4 H <= maxTBSize Horizontal If notabove cases and H > maxTBSize 4 Vertical If not above cases and H >maxTBSize 4

For each of these sub-partitions, a residual signal is generated byentropy decoding the coefficients sent by the encoder and then invertquantizing and invert transforming them. Then, the sub-partition isintra predicted and finally the corresponding reconstructed samples areobtained by adding the residual signal to the prediction signal.Therefore, the reconstructed values of each sub-partition will beavailable to generate the prediction of the next one, which will repeatthe process and so on. All sub-partitions share the same intra mode.

Hereinafter, inner sub-partition is used to represent sub-partitionsexcept the first sub-partition. If an ISP block is split in horizontal(vertical) direction, the first sub-partition means the above (left)sub-partition.

TABLE 2 Specification of trTypeHor and trTypeVer depending onpredModeIntra predModeIntra trTypeHor trTypeVer INTRA_PLANAR, (nTbW >= 4&& (nTbH >= 4 INTRA_ANGULAR31, nTbW <= 16) ? && INTRA_ANGULAR32,DST-VII: DCT-II nTbH <= 16) ? INTRA_ANGULAR34, DST-VII: INTRA_ANGULAR36,DCT-II INTRA_ANGULAR37 INTRA_ANGULAR33, DCT-II DCT-II INTRA_ANGULAR35INTRA_ANGULAR2, (nTbW >= 4 && DCT-II INTRA_ANGULAR4, . . . ,INTRA_ANGULAR28, nTbW <= 16) ? INTRA_ANGULAR30, DST-VII: DCT-IIINTRA_ANGULAR39, INTRA_ANGULAR41, . . . , INTRA_ANGULAR63,INTRA_ANGULAR65 INTRA_ANGULAR3, DCT-II (nTbH >= 4 INTRA_ANGULAR5, . . ., INTRA_ANGULAR27, && INTRA_ANGULAR29, nTbH <= 16) ? INTRA_ANGULAR38,DST-VII: INTRA_ANGULAR40, . . . , INTRA_ANGULAR64, DCT-IIINTRA_ANGULAR66

2.8. Affine Linear Weighted Intra Prediction (ALWIP, a.k.a. Matrix BasedIntra Prediction)

Affine linear weighted intra prediction (ALWIP, a.k.a. Matrix basedintra prediction (MIP)) is proposed in JVET-N0217.

2.8.1. Generation of the Reduced Prediction Signal by Matrix VectorMultiplication

The neighboring reference samples are firstly down-sampled via averagingto generate the reduced reference signal bdry_(red). Then, the reducedprediction signal pred_(red) is computed by calculating a matrix vectorproduct and adding an offset:pred_(red) =A·bdry_(red) +b.

Here, A is a matrix that has W_(red)·H_(red) rows and 4 columns if W=H=4and 8 columns in all other cases. b is a vector of size W_(red)·H_(red).

2.8.2. Illustration of the Entire ALWIP Process

The entire process of averaging, matrix vector multiplication and linearinterpolation is illustrated for different shapes in FIG. 8 , FIG. 9 ,FIG. 10 and FIG. 11 . Note, that the remaining shapes are treated as inone of the depicted cases.

-   -   1. Given a 4×4 block, ALWIP takes two averages along each axis        of the boundary. The resulting four input samples enter the        matrix vector multiplication. The matrices are taken from the        set S₀. After adding an offset, this yields the 16 final        prediction samples. Linear interpolation is not necessary for        generating the prediction signal. Thus, a total of        (4·16)/(4·4)=4 multiplications per sample are performed.    -   2. Given an 8×8 block, ALWIP takes four averages along each axis        of the boundary. The resulting eight input samples enter the        matrix vector multiplication. The matrices are taken from the        set S₁. This yields 16 samples on the odd positions of the        prediction block. Thus, a total of (8·16)/(8·8)=2        multiplications per sample are performed. After adding an        offset, these samples are interpolated vertically by using the        reduced top boundary. Horizontal interpolation follows by using        the original left boundary.    -   3. Given an 8×4 block, ALWIP takes four averages along the        horizontal axis of the boundary and the four original boundary        values on the left boundary. The resulting eight input samples        enter the matrix vector multiplication. The matrices are taken        from the set S₁. This yields 16 samples on the odd horizontal        and each vertical positions of the prediction block. Thus, a        total of (8·16)/(8·4)=4 multiplications per sample are        performed. After adding an offset, these samples are        interpolated horizontally by using the original left boundary.        The transposed case is treated accordingly.    -   4. Given a 16·16 block, ALWIP takes four averages along each        axis of the boundary. The resulting eight input samples enter        the matrix vector multiplication. The matrices are taken from        the set S₂. This yields 64 samples on the odd positions of the        prediction block. Thus, a total of (8·64)/(16·16)=2        multiplications per sample are performed. After adding an        offset, these samples are interpolated vertically by using eight        averages of the top boundary. Horizontal interpolation follows        by using the original left boundary. The interpolation process,        in this case, does not add any multiplications. Therefore,        totally, two multiplications per sample are required to        calculate ALWIP prediction.

For larger shapes, the procedure is essentially the same and it is easyto check that the number of multiplications per sample is less thanfour.

For W×8 blocks with W>8, only horizontal interpolation is necessary asthe samples are given at the odd horizontal and each vertical position.

Finally, for W×4 blocks with W>8, let A_(k) be the matrix that arises byleaving out every row that corresponds to an odd entry along thehorizontal axis of the down-sampled block. Thus, the output size is 32and again, only horizontal interpolation remains to be performed. Thetransposed cases are treated accordingly.

2.8.3. Adapted MPM-List Derivation for Conventional Luma and ChromaIntra-Prediction Modes

The proposed ALWIP-modes are harmonized with the MPM-based coding of theconventional intra-prediction modes as follows. The luma and chromaMPM-list derivation processes for the conventional intra-predictionmodes uses fixed tables map_alwip_to_angular_(idx), idx∈{0,1,2}, mappingan ALWIP-mode predmode_(ALWIP) on a given PU to one of the conventionalintra-prediction modespredmode_(Angular)=map_alwip_to_angular_(idx(PU))[predmode_(ALWIP)].

For the luma MPM-list derivation, whenever a neighboring luma block isencountered which uses an ALWIP-mode predmode_(ALWIP), this block istreated as if it was using the conventional intra-prediction modepredmode_(Angular). For the chroma MPM-list derivation, whenever thecurrent luma block uses an LWIP-mode, the same mapping is used totranslate the ALWIP-mode to a conventional intra prediction mode.

2.9. Quantized Residual Block Differential Pulse-Code Modulation(QR-BDPCM)

In JVET-M0413, a quantized residual block differential pulse-codemodulation (QR-BDPCM) is proposed to code screen contents efficiently.

The prediction directions used in QR-BDPCM can be vertical andhorizontal prediction modes. The intra prediction is done on the entireblock by sample copying in prediction direction (horizontal or verticalprediction) similar to intra prediction. The residual is quantized andthe delta between the quantized residual and its predictor (horizontalor vertical) quantized value is coded. This can be described by thefollowing: For a block of size M (rows)×N (cols), let r_(i,j), 0≤i≤M−1,0≤j≤N−1 be the prediction residual after performing intra predictionhorizontally (copying left neighbor pixel value across the thepredictedblock line by line) or vertically (copying top neighbor line toeach line in the predicted block) using unfiltered samples from above orleft block boundary samples. Let Q(r_(i,j)), 0≤i≤M−1, 0≤j≤N−1 denote thequantized version of the residual r_(i,j), where residual is differencebetween original block and the predicted block values. Then the blockDPCM is applied to the quantized residual samples, resulting in modifiedM×N array {tilde over (R)} with elements {tilde over (r)}_(i,j). Whenvertical BDPCM is signalled:

$\begin{matrix}{{\overset{˜}{r}}_{i,j} = \left\{ \begin{matrix}{{{Q\left( r_{i,j} \right)}\ ,}\ } & {{i = 0},\ {0 \leq j \leq \left( {N - 1} \right)}} \\{{{{Q\left( r_{i,j} \right)} - {Q\left( r_{{({i - 1})},j} \right)}}\ ,}\ } & {{1 \leq i \leq \left( {M - 1} \right)}\ ,\ {0 \leq j \leq \left( {N - 1} \right)}}\end{matrix} \right.} & \text{(2-7-1)}\end{matrix}$

For horizontal prediction, similar rules apply, and the residualquantized samples are obtained by

$\begin{matrix}{{\overset{˜}{r}}_{i,j} = \left\{ \begin{matrix}{{Q\left( r_{i,j} \right)}\ ,} & {{0 \leq i \leq \left( {M - 1} \right)},\ {j = 0}} \\{{{{Q\left( r_{i,j} \right)} - {Q\left( r_{i,{({j - 1})}} \right)}}\ ,}\ } & {{0 \leq i \leq \left( {M - 1} \right)}\ ,\ {1 \leq j \leq \left( {N - 1} \right)}}\end{matrix} \right.} & \text{(2-7-2)}\end{matrix}$

The residual quantized samples {tilde over (r)}_(i,j) are sent to thedecoder.

On the decoder side, the above calculations are reversed to produce Q(r_(i,j)), 0≤i≤M−1, 0≤j≤N−1. For vertical prediction case,Q(r _(i,j))=Σ_(k=0) ^(i) {tilde over (r)}_(k,j),0≤i≤(M−1),0≤j≤(N−1)  (2-7-3)

-   -   For horizontal case,        Q(r _(i,j))=Σ_(k=0) ^(j) {tilde over (r)}        _(i,k),0≤i≤(M−1),0≤j≤(N−1)  (2-7-4)

The inverse quantized residuals, Q⁻¹(Q(r_(i,j))), are added to the intrablock prediction values to produce the reconstructed sample values.

The main benefit of this scheme is that the inverse DPCM can be done onthe fly during coefficient parsing simply adding the predictor as thecoefficients are parsed or it can be performed after parsing.

2.10.Intra Chroma Coding

In addition to the intra chroma prediction modes, CCLM and joint chromaresidual coding are introduced in VVC.

2.10.1. Cross-Component Linear Model (CCLM)

To reduce the cross-component redundancy, a cross-component linear model(CCLM) prediction mode is used in the VTM4, for which the chroma samplesare predicted based on the reconstructed luma samples of the same CU byusing a linear model as follows:pred_(C)(i,j)=α·rec_(L)′(i,j)+β

where pred_(C)(i,j) represents the predicted chroma samples in a CU andrec_(L)(i,j) represents the downsampled reconstructed luma samples ofthe same CU. Linear model parameter α and β are derived from therelationship between luma values and chroma values from four samples atspecific positions. Among the four samples, the two larger values areaveraged, and the two smaller values are averaged. The averaged valuesare then utilized to derive the linear model parameters.

2.10.2. Joint Chroma Residual Coding

If chrominance reshaper is active, reshaping is applied to the receivedresidual identically to what is done in separate coding modes (that is,the joint residual signal is reshaped). On the encoder side the averageof positive Cb residual and negative Cr residual are used as the jointresidual when testing this mode:resJoint=(resCb−resCr)/2

One bin indicator is signaled in the bitstream to enable the mode. Inthe case the mode is enabled a joint residual signal is coded in thebitstream. On the decoder side the joint residual is used for Cbcomponent and a negative version of the residual is applied for Cr.

2.11.Sub-Block Transform (SBT) in VVC

For an inter-predicted CU with cu_cbf equal to 1, cu_sbt_flag may besignaled to indicate whether the whole residual block or a sub-part ofthe residual block is decoded. In the former case, inter MTS informationis further parsed to determine the transform type of the CU. In thelatter case, a part of the residual block is coded with inferredadaptive transform and the other part of the residual block is zeroedout. The SBT is not applied to the combined inter-intra mode, sincealmost no coding gain is achieved.

2.11.1. Sub-Block TU Tiling

When SBT is used for a inter CU, SBT type and SBT position informationare further decoded from the bitstream. There are two SBT types and twoSBT positions, as indicated in FIG. 12 . For SBT-V (or SBT-H), the TUwidth (or height) may equal to half of the CU width (or height) or ¼ ofthe CU width (or height), signaled by another flag, resulting in 2:2split or 1:3/3:1 split. The 2:2 split is like a binary tree (BT) splitwhile the 1:3/3:1 split is like an asymmetric binary tree (ABT) split.If one side of CU is 8 in luma samples, the 1:3/3:1 split along thisside is not allowed. Hence, there are at most 8 SBT modes for a CU.

Quad-tree (QT) split is further used to tile one CU into 4 sub-blocks,and still one sub-block has residual, as shown in FIG. 13 . This SBTtype is denoted as SBT-Q. This part was not adopted by VVC.

SBT-V, SBT-H and SBT-Q are allowed for CU with width and height both nolarger than maxSbtSize. The maxSbtSize is signaled in SPS. For HD and 4Ksequences, maxSbtSize is set as 64 by encoder; for other smallerresolution sequences, maxSbtSize is set as 32.

2.11.2. Transform Type of the Sub-Block

Position-dependent transform is applied on luma transform blocks inSBT-V and SBT-H (chroma TB always using DCT-2). The two positions ofSBT-H and SBT-V are associated with different core transforms. Morespecifically, the horizontal and vertical transforms for each SBTposition is specified in FIG. 12 . For example, the horizontal andvertical transforms for SBT-V position 0 is DCT-8 and DST-7,respectively. When one side of the residual TU is greater than 32, thecorresponding transform is set as DCT-2. Therefore, the sub-blocktransform jointly specifies the TU tiling, cbf, and horizontal andvertical transforms of a residual block, which may be considered asyntax shortcut for the cases that the major residual of a block is atone side of the block.

FIG. 12 is an illustration of sub-block transform modes SBT-V and SBT-H(The grey area is a TU which may have non-zero coefficients; the whitearea is a zero-out TU with all zero coefficients).

FIG. 13 is an illustration of sub-block transform modes SBT-Q.

2.12.Partition Tree

In VTM5, the coding tree scheme supports the ability for the luma andchroma to have a separate block tree structure. Currently, for P and Bslices, the luma and chroma CTBs in one CTU have to share the samecoding tree structure. However, for I slices, the luma and chroma canhave separate block tree structures. When separate block tree mode isapplied, luma CTB is partitioned into CUs by one coding tree structure,and the chroma CTBs are partitioned into chroma CUs by another codingtree structure. This means that a CU in an I slice may consist of acoding block of the luma component or coding blocks of two chromacomponents, and a CU in a P or B slice always consists of coding blocksof all three colour components unless the video is monochrome.

3. EXAMPLES OF TECHNICAL PROBLEMS ADDRESSED BY THE SOLUTIONS DESCRIBEDIN THIS DOCUMENT

How to apply ACT to the VVC design needs to be studied, especially theinteraction between ACT and other tools needs to be resolved:

-   1. ILR is to convert the luma component from a whole    picture/slice/tile from the original domain to reshaped domain and    code everything in the reshaped domain. However, for the chroma    component, it is coded in the original domain. ACT requires to get    the residual signal of pixels for three-color components.-   2. How to handle ACT when dual tree is enabled.

4. EXAMPLES OF SOLUTIONS AND EMBODIMENTS

The listing below should be considered as examples to explain generalconcepts. These inventions should not be interpreted in a narrow way.Furthermore, these techniques can be combined in any manner.

In the following discussion, a CU may comprise information associated toall the three-color components with the single tree coding structure. Ora CU may comprise information only associated to the luma colorcomponent with the mono-color coding. Or a CU may comprise informationonly associated to the luma color component (e.g., Y component in YCbCrformat or G component in GBR format) with the dual tree codingstructure. Or a CU may comprise information only associated to the twochroma components (e.g., Cb and Cr components in YCbCr format or B and Rcomponents in GBR format) with the dual-tree coding structure.

In the following description, a “block” may refer to coding unit (CU) ora transform unit (TU) or any rectangle or polygonal region of videodata. a “current block” may refer to a current being decoded/codedcoding unit (CU) or a current being decoded/coded transform unit (TU) orany being decoded/coded coding rectangle region of video data. “CU” or“TU” may be also known as “coding block” and “transform block”.

In the following discussions, the term ‘ACT’ may represent anytechnology that may convert the original signals/predictionsignals/reconstructed signals/residual signals of three-color componentsfrom one domain to another domain, not necessarily to be the same designin HEVC SCC.

-   1. ACT is disabled for all blocks in a video unit when dual tree    partitioning structure is enabled for the video unit (e.g.,    slice/tile/brick/picture/a region covering one or multiple CTUs).    -   a. Indications of usage of ACT may be conditionally signaled        based on the usage of the dual-tree partitioning structure.        -   i. In one example, when the dual-tree partitioning structure            is applied, the signaling of indications of usage of ACT may            be skipped.-   2. ACT is disabled for all blocks in a video unit when ILR is    enabled for the video unit (e.g, slice/tile/brick/picture/a region    covering one or multiple CTUs).    -   a. Indications of usage of ACT may be conditionally signaled        based on the usage of ILR.        -   i. In one example, when ILR is applied, the signaling of            indications of usage of ACT may be skipped.-   3. ACT and a coding tool X are exclusively applied for a video block    (e.g., CU/TU).    -   a. In one example, X is CCLM. If CCLM is enabled for chroma        components of the video block, ACT is disabled; and/or vice        versa.    -   b. In one example, X is joint chroma residual coding. If joint        chroma residual coding is enabled for chroma components of the        video block, ACT is disabled; and/or vice versa.    -   c. In one example, X is matrix based intra prediction method. If        the matrix based intra prediction method is enabled for the luma        component of the video block, ACT is disabled; and/or vice        versa.    -   d. In one example, X is QR-BDPCM. If QR-BDPCM is enabled for the        luma components of the video block, ACT is disabled; and/or vice        versa.    -   e. In one example, X is sub-block transform (SBT). If SBT is        enabled for the luma components of the video block, ACT is        disabled; and/or vice versa.    -   f. In one example, X is multiple transform selection (MTS). If        MTS is enabled for the luma components of the video block, ACT        is disabled; and/or vice versa.    -   g. In one example, X is Low frequency non-separable transform        (LFNST). If LFNST is enabled, ACT is disabled; and/or vice        versa.    -   h. In one example, X is Pulse Code Modulation (PCM). If PCM is        enabled, ACT is disabled; and/or vice versa.    -   i. In one example, X is Transform Skip (TS). If TS is enabled,        ACT is disabled; and/or vice versa.    -   j. In one example, X is Intra Subblock Partitioning (ISP). If        ISP is enabled, ACT is disabled; and/or vice versa.    -   k. Alternatively, furthermore, indications of usage of ACT may        be conditionally signaled based on the usage of the coding tool        X.        -   i. In one example, when the coding tool X is enabled, the            signaling of indications of usage of ACT may be skipped.    -   l. Alternatively, furthermore, indications of usage of tool X        may be conditionally signaled based on the usage of ACT.        -   i. In one example, when ACT is enabled, the signaling of            indications of usage of the coding tool X may be skipped.    -   m. Alternatively, the above mentioned tools and ACT may be both        enabled for one video block.-   4. ACT and dual tree partition structure may be both enabled for one    video unit (e.g., picture/slice/tile/brick)    -   a. Alternatively, furthermore, the signaling of usage of dual        tree partition structure is moved from video unit-level to video        block (e.g., CTU/CTB or VPDU)-level.    -   b. ACT and dual tree partition structure may be both enabled for        one video block        -   i. In one example, at the encoder side, ACT may be firstly            applied to a CTU/CTB before partitioning of the CTU/CTB.        -   ii. In one example, at the decoder side, a CTU/CTB may be            first decoded, followed by inverse color-space transform.-   5. ACT and ILR may be both enabled for one video unit (e.g.,    picture/slice/tile/brick)    -   a. Alternatively, furthermore, the signaling of usage of ILR is        moved from video unit-level to video block (e.g., CU/TU)-level.    -   b. ACT and ILR may be both enabled for one video block (e.g.,        CU/TU).        -   i. In one example, at the encoder side, ACT may be firstly            applied, followed by ILR. That is, the prediction signal and            residual signal is firstly generated in the original domain,            ACT is applied to convert the residual signal from the            original domain to a different color-space domain; and ILR            is further applied to convert the residual signal to the            reshaped domain.        -   ii. In one example, at the decoder side, ILR may be firstly            applied, followed by inverse color-space transform. That is,            ILR is firstly applied to convert the decoded residual            signal from the reshaped domain to the color-space domain;            then ACT is applied to convert from the color-space domain            to the original domain.-   6. ACT and SBT may be both enabled for one video block (e.g.,    CU/TU).    -   a. In one example, the predictor error in a converted        color-space domain (e.g., original domain is RGB, converted        domain is YCoCg with ACT) is coded with two TUs. One of them is        all zero coefficients and the other one has non-zero        coefficients.        -   i. Alternatively, furthermore, the one TU that has non-zero            coefficients may be obtained via transforms or transform            skip.        -   ii. In one example, how to split it to 2 TUs; and/or what            kinds of transforms may be applied to one of the two TUs may            be signalled, e.g., in a similar way as SBT.-   7. For a video unit (e.g., slice/tile/brick/picture), ACT may be    enabled in different levels, such as CU-level and TU-level.    -   a. In one example, the signaling of usage of ACT may be in        different levels, such as CU-level and TU-level, for different        video blocks in the video unit.    -   b. Whether to enable/signal the ACT in CU or TU level and/or        whether to signal the usage of ACT may be determined based on        coding characteristics.        -   i. In one example, whether to enable/signal the ACT in CU or            TU level may be determined based on the dimensions of the            current CU. Suppose the width and height of the current CU            are denoted as W and H, respectively.            -   1. For example, whether to enable/signal the ACT in CU                or TU level may be determined based on whether the                current CU has a size greater than the VPDU size.                -   a. In one example, if current CU has a size greater                    than the VPDU size, CU-level signaling/usage of ACT                    may be applied (i.e., all TUs share the same on/off                    control of ACT). Otherwise, TU-level signaling/usage                    of ACT may be applied.                -   b. Alternatively, if current CU has a size greater                    than the VPDU size, TU-level signaling/usage of ACT                    may be applied (i.e., all TUs share the same on/off                    control of ACT). Otherwise, CU-level signaling/usage                    of ACT may be applied.                -   c. In one example, if current CU has a size greater                    than the VPDU size, ACT is disabled without being                    signaled.            -   2. In one example, whether to enable/signal the ACT in                CU or TU level may be determined based on the current CU                has a size greater than the maximum TU size.            -   3. In one example, ACT is disabled when W>=T1 and H>=T2.                E.g. T1=T2=32.                -   a. Alternatively, ACT is disabled when W>=T1 or                    H>=T2. E.g. T1=T2=32.                -   b. Alternatively, ACT is disabled when W<=T1 and                    H<=T2. E.g. T1=T2=8.                -   c. Alternatively, ACT is disabled when W<=T1 or                    H<=T2. E.g. T1=T2=8.                -   d. Alternatively, ACT is disabled when W*H>=T. E.g.                    T=1024.                -   e. Alternatively, ACT is disabled when W*H<=T. E.g.                    T=64.        -   ii. In one example, whether to enable/signal the ACT in CU            or TU level may be determined based on the current CU is            coded with sub-block partition tools, such as ISP.

FIG. 14 is a block diagram of a video processing apparatus 1400. Theapparatus 1400 may be used to implement one or more of the methodsdescribed herein. The apparatus 1400 may be embodied in a smartphone,tablet, computer, Internet of Things (IoT) receiver, and so on. Theapparatus 1400 may include one or more processors 1402, one or morememories 1404 and video processing hardware 1406. The processor(s) 1402may be configured to implement one or more methods described in thepresent document. The memory (memories) 1404 maybe used for storing dataand code used for implementing the methods and techniques describedherein. The video processing hardware 1406 may be used to implement, inhardware circuitry, some techniques described in the present document.

FIG. 15 is another example of a block diagram of a video processingsystem in which disclosed techniques may be implemented. FIG. 15 is ablock diagram showing an example video processing system 1510 in whichvarious techniques disclosed herein may be implemented. Variousimplementations may include some or all of the components of the system1510. The system 1510 may include input 1512 for receiving videocontent. The video content may be received in a raw or uncompressedformat, e.g., 8 or 10 bit multi-component pixel values, or may be in acompressed or encoded format. The input 1512 may represent a networkinterface, a peripheral bus interface, or a storage interface. Examplesof network interface include wired interfaces such as Ethernet, passiveoptical network (PON), etc. and wireless interfaces such as Wi-Fi orcellular interfaces.

The system 1510 may include a coding component 1514 that may implementthe various coding or encoding methods described in the presentdocument. The coding component 1514 may reduce the average bitrate ofvideo from the input 1512 to the output of the coding component 1514 toproduce a coded representation of the video. The coding techniques aretherefore sometimes called video compression or video transcodingtechniques. The output of the coding component 1514 may be eitherstored, or transmitted via a communication connected, as represented bythe component 1516. The stored or communicated bitstream (or coded)representation of the video received at the input 1512 may be used bythe component 1518 for generating pixel values or displayable video thatis sent to a display interface 1520. The process of generatinguser-viewable video from the bitstream representation is sometimescalled video decompression. Furthermore, while certain video processingoperations are referred to as “coding” operations or tools, it will beappreciated that the coding tools or operations are used at an encoderand corresponding decoding tools or operations that reverse the resultsof the coding will be performed by a decoder.

Examples of a peripheral bus interface or a display interface mayinclude universal serial bus (USB) or high definition multimediainterface (HDMI) or Displayport, and so on. Examples of storageinterfaces include SATA (serial advanced technology attachment), PCI,IDE interface, and the like. The techniques described in the presentdocument may be embodied in various electronic devices such as mobilephones, laptops, smartphones or other devices that are capable ofperforming digital data processing and/or video display.

In some embodiments, the video coding methods may be implemented usingan apparatus that is implemented on a hardware platform as describedwith respect to FIG. 14 or 15 .

FIG. 16A is a flowchart of an example method 1610 of video processing.The method 1610 includes, at step 1612, determining, due to a dual treepartitioning structure being used for a conversion between a video unitand a coded representation of the video unit, that use of an adaptivecolor space transformation (ACT) tool is disabled for the video unit.The method 1610 further includes, at step 1614, performing, based on thedetermining, the conversion by disabling the ACT tool for the videounit. In some implementations of the present method 1610 and otherexamples methods, use of the ACT tool comprises: converting, duringencoding, a representation of a visual signal from a first color domainto a second color domain, or converting, during decoding, arepresentation of a visual signal from the second color domain to thefirst color domain.

FIG. 16B is a flowchart of an example method 1620 of video processing.The method 1620 includes, at step 1622, determining that a dual treepartitioning structure and an adaptive color space transformation (ACT)tool are used for a conversion between a video unit and a codedrepresentation of the video unit. The method 1620 further includes, atstep 1624, performing based on the determining, the conversion byenabling the ACT tool for the video unit.

FIG. 17A is a flowchart of an example method 1710 of video processing.The method 1710 includes, at step 1712, determining, for a conversionbetween a current videoblock of a video and a coded representation ofthe video, that applicability of a first coding tool and a second codingtool is mutually exclusive. The method 1710 further includes, at step1714, performing the conversion based on the determining.

FIG. 17B is a flowchart of an example method 1720 of video processing.The method 1720 includes, at step 1712, determining that both a codingtool and an adaptive color space transformation (ACT) tool are used fora conversion between a current video block of a video and a codedrepresentation of the video. The method 1720 further includes, at step1724, performing based on the determining, the conversion by enablingthe ACT tool for the current video block.

FIG. 17C is a flowchart of an example method 1730 of video processing.The method 1730 includes, at step 1732, determining, for a conversionbetween a current video block of a video unit of a video and a codedrepresentation of the video, that an adaptive color space transformation(ACT) tool is disabled for the conversion due to an in-loop reshaping(ILR) tool being enabled for the video unit. The method 1730 furtherincludes, at step 1734, performing, based on the determining, theconversion. In some implementations of the present method 1730 and otherexamples methods, the use of the ILR tool includes constructing thevideo unit based on a luma reshaping between a first domain and a seconddomain and/or a chroma residue scaling in a luma-dependent manner.

FIG. 17D is a flowchart of an example method 1740 of video processing.The method 1740 includes, at step 1742, determining that both an in-loopreshaping (ILR) tool and an adaptive color space transformation (ACT)tool are enabled for a conversion between a video unit and a codedrepresentation of the video unit. The method 1740 further includes, atstep 1744, performing based on the determining, the conversion.

FIG. 17E is a flowchart of an example method 1750 of video processing.The method 1750 includes, at step 1752, determining that both asub-block transform (SBT) tool and an adaptive color spacetransformation (ACT) coding tool are enabled for a conversion between acurrent video block and a coded representation of the current videoblock. The method 1750 further includes, at step 1754, performing, basedon the determining, the conversion. In some implementations of thepresent method 1750 and other example methods, use of the SBT toolcomprises applying a transform process or an inverse transform processon a sub-part of a prediction residual block.

FIG. 18 is a flowchart of an example method 1800 of video processing.The method 1800 includes, at step 1810, performing a conversion betweena video unit of a video and a coded representation of the video, wherethe video unit comprises one or more partitions at a first levelcomprising one or more partitions at a second level. In someimplementations, the coded representation conforms to a formatting rule,wherein the formatting rule specifies whether to include, or a partitionlevel at which a syntax element indicative of use of an adaptive colorspace transformation (ACT) tool for representing the one or more secondlevel partitions in the coded representation is included in the codedrepresentation, wherein the partition level is one of the first level,the second level or the video unit.

Some embodiments of the disclosed technology include making a decisionor determination to enable a video processing tool or mode. In anexample, when the video processing tool or mode is enabled, the encoderwill use or implement the tool or mode in the processing of a block ofvideo, but may not necessarily modify the resulting bitstream based onthe usage of the tool or mode. That is, a conversion from the block ofvideo to the bitstream representation of the video will use the videoprocessing tool or mode when it is enabled based on the decision ordetermination. In another example, when the video processing tool ormode is enabled, the decoder will process the bitstream with theknowledge that the bitstream has been modified based on the videoprocessing tool or mode. That is, a conversion from the bitstreamrepresentation of the video to the block of video will be performedusing the video processing tool or mode that was enabled based on thedecision or determination.

Some embodiments of the disclosed technology include making a decisionor determination to disable a video processing tool or mode. In anexample, when the video processing tool or mode is disabled, the encoderwill not use the tool or mode in the conversion of the block of video tothe bitstream representation of the video. In another example, when thevideo processing tool or mode is disabled, the decoder will process thebitstream with the knowledge that the bitstream has not been modifiedusing the video processing tool or mode that was disabled based on thedecision or determination.

In the present document, the term “video processing” may refer to videoencoding video decoding, video compression or video decompression. Forexample, video compression algorithms may be applied during conversionfrom pixel representation of a video to a corresponding bitstreamrepresentation or vice versa. The bitstream representation of a currentvideo block may, for example, correspond to bits that are eitherco-located or spread in different places within the bitstream, as isdefined by the syntax. For example, a macroblock may be encoded in termsof transformed and coded error residual values and also using bits inheaders and other fields in the bitstream.

Various solutions and embodiments described in the present document arefurther described using a list of clauses. The first set of clausesdescribe certain features and aspects of the disclosed techniques in theprevious section.

Item 1 in previous section provides additional examples of the followingclauses.

1. A method of video processing, comprising: determining that due to adual tree partitioning structure being used for a conversion between avideo unit and a bitstream representation of the video unit, that use ofadaptive color space transformation (ACT) is disabled for the videounit; and performing, based on the determining, the conversion bydisabling the ACT for the vide unit.

2. The method of clause 1, wherein the bitstream representation excludesbits for providing information about usage of the ACT.

Item 2 in previous section provides additional examples of the followingclauses.

1. A method of video processing, comprising: determining, due to anin-loop reshaper being enabled for use for a conversion between a videounit and a bitstream representation of the video unit, to disable anadaptive color space transformation (ACT) for the conversion; andperforming, based on the determining, the conversion by disabling theACT for the video unit.

2. The method of clause 1, wherein the bitstream representation includesbits for providing information about usage of the ILR.

Item 3 in previous section provides additional examples of the followingclauses.

3. A method of video processing. comprising: determining, due to acoding tool being used for a conversion between a video unit and abitstream representation of the video unit, that an adaptive color spacetransformation tool is disabled for the conversion; and performing,based on the determining, the conversion by disabling the ACT for thevideo unit.

4. The method of clause 3, wherein the coding tool corresponds to across-component linear model tool.

5. The method of clause 3, wherein the coding tool corresponds to ajoint chroma residual coding.

6. The method of clause 3, wherein the coding tool corresponds to amultiple transform selection (MTS) coding tool.

Item 4 in previous section provides additional examples of the followingclauses.

7. A method of video processing, comprising: determining that both adual tree partitioning structure and an adaptive color spacetransformation (ACT) coding tool are used for a conversion between avideo unit and a bitstream representation of the video unit; andperforming based on the determining, the conversion by enabling the ACTfor the video unit.

8. The method of clause 7, wherein the bitstream representation includessignaling of the dual partition tree structure at a video block levelthat is a finer level than the video unit level.

Item 5 in previous section provides additional examples of the followingclauses.

9. A method of video processing, comprising: determining that both anin-loop reshaping (ILR) and an adaptive color space transformation (ACT)coding tool are used for a conversion between a video unit and abitstream representation of the video unit; and performing based on thedetermining, the conversion using the ILR and the ACT coding tool.

10. The method of clause 9, wherein the performing the conversionincludes, during encoding, first applying the ACT coding tool and nextapplying the ILR on a result of the applying the ACT.

11. The method of clause 9, wherein the performing the conversionincludes, first applying the ILR and then an inverse color spacetransform is applied to a result of the ILR.

Item 6 in previous section provides additional examples of the followingclauses.

12. A method of video processing, comprising: determining that both an(SBT) and an adaptive color space transformation (ACT) coding tool areused for a conversion between a video unit and a bitstreamrepresentation of the video unit; and performing, based on thedetermining, the conversion using the SBT and the ACT coding tool.

13. The method of clause 12, wherein a prediction error during theconversion is transformed from an RGB color space to a YCoCg colorspace.

14. The method of clause 13, wherein the prediction error, after thetransforming, is coded using at least two transform uint (TU)partitions.

Item 7 in previous section provides additional examples of the followingclauses.

15. The method of any of above clauses, wherein the video unit comprisesslice or a tile or a brick or a picture.

16. The method of clause 15, wherein the determining is performed at asub-video unit level, wherein the sub-unit level corresponds to a codingunit (CU) or a transform unit (TU).

17. The method of clause 16, wherein the determining at the sub-videounit level is based on coding characteristics at the sub-unit level.

18. The method of clause 17, wherein the coding characteristics includea size of the CU and/or a size of the TU and/or a relationship betweenthe size of the CU and the size of the TU.

19. The method of any of above clauses, wherein the conversion includesdecoding the bitstream representation to generate the video unit.

20. The method of any of above clauses, wherein the conversion includesencoding the video unit into the bitstream representation.

21. A video encoder apparatus comprising a processor configured toimplement a method recited in any one or more of above clauses.

22. A video decoder apparatus comprising a processor configured toimplement a method recited in any one or more of above clauses.

23. A computer-readable medium having code for implementing a methoddescribed in any one or more of above clauses stored thereupon.

The second set of clauses describe certain features and aspects of thedisclosed techniques in the previous section, for example, ExampleImplementations 1 and 4.

1. A method of video processing, comprising: determining, due to a dualtree partitioning structure being used for a conversion between a videounit and a coded representation of the video unit, that use of anadaptive color space transformation (ACT) tool is disabled for the videounit; and performing, based on the determining, the conversion bydisabling the ACT tool for the video unit, wherein the use of the ACTtool comprises: converting, during encoding a representation of a visualsignal from a first color domain to a second color domain, or convertingduring decoding, a representation of a visual signal from the secondcolor domain to the first color domain.

2. The method of clause 1, wherein the visual signal comprises anoriginal signal, a prediction signal, a reconstructed signal or aresidual signal.

3. The method of clause 1, wherein the video unit corresponds to aslice, a tile, a brick, a picture, or a video region covering one ormore coding tree units.

4. The method of clause 1, wherein the ACT tool is disabled for allcoding blocks in the video unit and the coding blocks are generatedbased on the dual tree partitioning structure.

5. The method of clause 1, wherein the dual tree partitioning structureuses separate partition trees for luma and chroma components.

6. The method of clause 1, wherein the use of the ACT tool is signaledbased on use of the dual tree partitioning structure.

7. The method of clause 1, wherein the coded representation excludesbits for providing information about the use of the ACT tool due tousage of the dual tree partitioning structure.

8. A method of video processing, comprising: determining that a dualtree partitioning structure and an adaptive color space transformation(ACT) tool are used for a conversion between a video unit and a codedrepresentation of the video unit; and performing, based on thedetermining, the conversion by enabling the ACT tool for the video unit,wherein use of the ACT tool comprises: converting, during encoding, arepresentation of a visual signal from a first color domain to a secondcolor domain, or converting, during decoding, a representation of avisual signal from the second color domain to the first color domain.

9. The method of clause 8, wherein the visual signal comprises anoriginal signal, a prediction signal, a reconstructed signal or aresidual signal.

10. The method of clause 8, wherein the video unit corresponds to aslice, a tile, a brick, a picture, or a vide region covering one or morecoding tree units.

11. The method of clause 8, wherein the coded representation includessignaling of the dual partition tree structure at a video block levelthat is a finer level than the video unit level.

12. The method of clause 8, wherein both the dual tree partitioningstructure and the ACT tool are enabled for a video block of the videounit.

13. The method of clause 12, wherein the performing the conversionincludes, during encoding, applying the ACT tool to the video blockprior to a partitioning of the video block.

14. The method of clause 12, wherein the performing the conversionincludes decoding the vide block and then performing an inverse colorspace transform on a result of the decoding.

15. The method of any of clauses 1 to 14, wherein the conversionincludes decoding the coded representation to generate the video unit.

16. The method of any of clauses 1 to 14, wherein the conversionincludes encoding the video unit into the coded representation.

17. A video processing apparatus comprising a processor configured toimplement a method recited in any one or more of clauses 1 to 16.

18. A computer readable medium storing program code that, when executed,causes a processor to implement a method recited in any one or more ofclauses 1 to 16.

The third set of clauses describe certain features and aspects of thedisclosed techniques in the previous section, for example, ExampleImplementations 2, 3, 5 and 6.

1. A video processing method, comprising: determining, for a conversionbetween a current video block of a video and a coded representation ofthe video, that applicability of a first coding tool and a second codingtool is mutually exclusive; and performing the conversion based on thedetermining, wherein the first coding tool corresponds to an adaptivecolor space transformation (ACT) tool; wherein use of the ACT toolcomprises: converting, during encoding a representation of a visualsignal from a first color domain to a second color domain, orconverting, during decoding, a representation of a visual signal fromthe second color domain to the first color domain.

2. The method of clause 1, wherein the visual signal comprises anoriginal signal, a prediction signal, a reconstructed signal or aresidual signal.

3. The method of clause 1, wherein the second coding tool corresponds toa cross-component linear model tool that uses a linear mode to deriveprediction values of a chroma component from another component.

4. The method of clause 1, wherein the second coding tool corresponds toa joint chroma residual coding tool in which prediction residual blockof two color components are jointly processed.

5. The method of clause 1, wherein the second coding tool corresponds toa matrix based intra prediction tool that includes generating aprediction signal based on predefined matrices and samples obtainedalong two axes of boundaries of the current video block.

6. The method of clause 1, wherein the second coding tool corresponds toa quantized residual block differential pulse-code modulation (QR-BDPCM)tool that includes coding residual differences between quantizedresidual and prediction residual into the coded representation orderiving quantized residual from the residual differences included inthe coded representation.

7. The method of clause 1, wherein the second coding tool corresponds toa sub-block transform (SBT) tool in which the current video block issplit into multiple sub-blocks and a transform or an inverse transformis only performed on a part of sub-blocks.

8. The method of clause 1, wherein the second coding tool corresponds toa multiple transform selection (MTS) tool that selects, for the currentvideo block, a transform among multiple transforms.

9. The method of clause 1, wherein the second coding tool corresponds toa low frequency non-separable transform (LFNST) tool that includesapplying, during encoding, a forward secondary transform to an output ofa forward primary transform applied to a residual of a video block priorto quantization, or includes applying, during decoding, an inversesecondary transform to an output of dequantization to the video blockbefore applying an inverse primary transform

10. The method of clause 1, wherein the second coding tool correspondsto a pulse code modulation (PCM) tool that digitally represents asampled analog signal.

11. The method of clause 1, wherein the second coding tool correspondsto a transform skip (TS) mode in which a transform is bypassed or anidentify transform is applied

12. The method of clause 1, wherein the second coding tool correspondsto an intra subblock partitioning (ISP) tool that includes dividing lumaintra-predicted blocks vertically or horizontally into sub-partitions.

13. The method of clause 1, wherein the use of the first coding tool issignaled based on use of the second coding tool.

14. The method of clause 13, wherein the coded representation excludesbits for providing information about the use of the first coding tooldue to usage of the second coding tool.

15. The method of clause 1, wherein use of the second coding tool issignaled based on the use of the first coding tool.

16. The method of clause 15, wherein the coded representation excludesbits for providing information about the use of the second coding tooldue to usage of the first coding tool.

17. A method of video processing, comprising: determining that both acoding tool and an adaptive color space transformation (ACT) tool areused for a conversion between a current video block of a video and acoded representation of the video; and performing, based on thedetermining, the conversion by enabling the ACT tool for the currentvideo block, wherein use of the ACT tool comprises: converting, duringencoding, a representation of a visual signal from a first color domainto a second color domain, or converting, during decoding, arepresentation of a visual signal from the second color domain to thefirst color domain.

18. The method of clause 17, wherein the coding tool comprises a matrixbased intra prediction (MIP) tool, a sub-block transform (SBT) tool, amultiple transform selection (MTS), a low frequency non-separabletransform (LFNST) tool, or a transform skip (TS) tool.

19. A method of video processing, comprising: determining, for aconversion between a current video block of a video unit of a video anda coded representation of the video, that an adaptive color spacetransformation (ACT) tool is disabled for the conversion due to anin-loop reshaping (ILR) tool being enabled for the video unit; andperforming, based on the determining, the conversion, and wherein theuse of the ILR tool includes constructing the video unit based on a lumareshaping between a first domain and a second domain and/or a chromaresidue scaling in a luma-dependent manner, and wherein use of the ACTtool comprises: converting, during encoding, a representation of avisual signal from a first color domain to a second color domain, orconverting, during decoding, a representation of a visual signal fromthe second color domain to the first color domain.

20. The method of clause 19, wherein the video unit corresponds to aslice, a tile, a brick, a picture, or a video region covering one ormore coding tree units.

21. The method of clause 19, wherein the use of the ACT tool is signaledbased on the use of the ILR tool.

22. The method of clause 19, wherein the coded representation excludesbits for providing information about the use of the ACT due to usage ofthe ILR.

23. A method of video processing, comprising: determining that both anin-loop reshaping (ILR) tool and an adaptive color space transformation(ACT) tool are enabled for a conversion between a video unit and a codedrepresentation of the video unit; and performing based on thedetermining, the conversion, and wherein use of the ILR tool includesconstructing the current video unit based on a first domain and a seconddomain and/or scaling chroma residue in a luma-dependent manner, andwherein use of the ACT tool comprises: converting, during encoding, arepresentation of a visual signal from a first color domain to a secondcolor domain, or converting, during decoding, a representation of avisual signal from the second color domain to the first color domain.

24. The method of clause 23, wherein the video unit corresponds to aslice, a tile, a brick, a picture, or a video region covering one ormore coding tree units.

25. The method of clause 23, wherein the coded representation includessignaling of the dual partition tree structure at a video block levelthat is a finer level than the video unit level.

26. The method of clause 23, wherein both the ILR tool and the ACT toolare enabled for a video block of the video unit.

27. The method of clause 26, wherein the performing the conversionincludes, during encoding, first applying the ACT tool to the videoblock and next applying the ILR tool on a result of the applying the ACTtool.

28. The method of clause 26, wherein the performing the conversionincludes first applying the ILR tool to the video block and thenapplying an inverse color space transform on a result of the ILR tool.

29. A method of video processing, comprising: determining that both asub-block transform (SBT) tool and an adaptive color spacetransformation (ACT) coding tool are enabled for a conversion between acurrent video block and a coded representation of the current videoblock; and performing, based on the determining, the conversion, whereinuse of the SBT tool comprises applying a transform process or an inversetransform process on a sub-part of a prediction residual block, andwherein use of the ACT tool comprises: converting, during encoding, arepresentation of a visual signal from a first color domain to a secondcolor domain, or converting, during decoding, a representation of avisual signal from the second color domain to the first color domain.

30. The method of clause 29, wherein a prediction error is convertedfrom an RGB color space to a YCoCg color space before the transformprocess in encoding or converted from a YCoCg color space to an RGBcolor space after the inverse transform process in decoding.

31. The method of clause 30, wherein the prediction error is coded usingtwo transform units (TUs) including a first transform unit (TU) and asecond transform unit (TU).

32. The method of clause 31, wherein the first transform unit (TU) hasall zero coefficient and the second transform unit (TU) has non-zerocoefficients.

33. The method of clause 32, wherein the second transform unit (TU) isobtained by performing a transform or a transform skip.

34. The method of clause 31, wherein the coded representation includesinformation as to how to split the current video block to the twotransform units (TUs) and/or types of transforms applied to at least oneof the two transform units (TUs).

35. The method of any of clauses 1 to 34, wherein the conversionincludes generating the video unit or the current video block from thecoded representation.

36. The method of any of clauses 1 to 34, wherein the conversionincludes generating the coded representation from the video unit or thecurrent video block.

37. A video processing apparatus comprising a processor configured toimplement a method recited in any one or more of clauses 1 to 36.

38. A computer readable medium storing program code that, when executed,causes a processor to implement a method recited in any one or more ofclauses 1 to 36.

The fourth set of clauses describe certain features and aspects of thedisclosed techniques in the previous section, for example, ExampleImplementation 7.

1. A video processing method, comprising: performing a conversionbetween a video unit of a video and a coded representation of the video,where the video unit comprises one or more partitions at a first levelcomprising one or more partitions at a second level, wherein the codedrepresentation conforms to a formatting rule, wherein the formattingrule specifies whether to include, or a partition level at which asyntax element indicative of use of an adaptive color spacetransformation (ACT) tool for representing the one or more second levelpartitions in the coded representation is included in the codedrepresentation, wherein the partition level is one of the first level,the second level or the video unit.

2. The method of clause 1, wherein the video unit corresponds to a sliceor a tile or a brick or a picture.

3. The method of clause 1, wherein the use of the ACT tool comprises:converting during encoding, a representation of a visual signal from afirst color domain to a second color domain, or converting, duringdecoding, a representation of a visual signal from the second colordomain to the first color domain.

4. The method of any of clauses 1 to 3, wherein the formatting rulespecifies that different levels for different video blocks in the videounit.

5. The method of clause 4, wherein the different levels correspond tothe coding unit (CU) and/or the transform unit (TU).

6. The method of any of clauses 1 to 5, wherein the formatting rulespecifies the partition level based on coding characteristics of theconversion.

7. The method of clause 6, wherein the coding characteristics include awidth (W) and/or a height (H) of a current coding unit.

8. The method of clause 7, wherein the formatting rule specifies thepartition level based on whether the current coding unit has a sizegreater than a size of a virtual pipelining data unit (VPDU).

9. The method of clause 8, wherein the partition level corresponds to acoding unit (CU) level in a case that the current coding unit has a sizegreater than the size of the VPDU.

10. The method of clause 8, wherein the partition level corresponds atransform unit (TU) level in a case that the current coding unit has asize greater than the size of the VPDU.

11. The method of clause 8, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatthe current coding unit has a size greater than the size of the VPDU.

12. The method of clause 7, wherein the formatting rule specifies thepartition level based on whether the current coding unit has a sizegreater than a maximum size of the transform unit (TU).

13. The method of clause 1, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatW>=T1 and H>=T2, W and H corresponding to a width and a height of acurrent coding unit, respectively.

14. The method of clause 1, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatW>=T1 or H>=T2, W and H corresponding to a width and a height of acurrent coding unit, respectively.

15. The method of clause 7, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatW<=T1 and H W and H corresponding to a width and a height of a currentcoding unit, respectively.

16. The method of clause 7, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatW<=T1 or H W and H corresponding to a width and a height of a currentcoding unit, respectively.

17. The method of clause 7, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatW*H>=T, W and H corresponding to a width and a height of a currentcoding unit, respectively.

18. The method of clause 7, wherein the formatting rule specifies not toinclude the syntax element due to the ACT tool disabled in a case thatW*H<=T, W and H corresponding to a width and a height of a currentcoding unit, respectively.

19. The method of clause 7, wherein the formatting rule specifies thepartition level based on whether the current coding unit is coded with asub-block partition tool.

20. The method of any of clauses 1 to 19, wherein the conversionincludes generating the video unit from the coded representation.

21. The method of any of clauses 1 to 19, wherein the conversionincludes generating the coded representation from the video unit.

22. A video processing apparatus comprising a processor configured toimplement a method recited in any one or more of clauses 1 to 21.

23. A computer readable medium storing program code that, when executed,causes a processor to implement a method recited in any one or more ofclauses 1 to 21.

From the foregoing, it will be appreciated that specific embodiments ofthe presently disclosed technology have been described herein forpurposes of illustration, but that various modifications may be madewithout deviating from the scope of the invention. Accordingly, thepresently disclosed technology is not limited except as by the appendedclaims.

Implementations of the subject matter and the functional operationsdescribed in this patent document can be implemented in various systems,digital electronic circuitry, or in computer software, firmware, orhardware, including the structures disclosed in this specification andtheir structural equivalents, or in combinations of one or more of them.Implementations of the subject matter described in this specificationcan be implemented as one or more computer program products, i.e., oneor more modules of computer program instructions encoded on a tangibleand non-transitory computer readable medium for execution by, or tocontrol the operation of, data processing apparatus. The computerreadable medium can be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more of them. The term “data processing unit” or “dataprocessing apparatus” encompasses all apparatus, devices, and machinesfor processing data, including by way of example a programmableprocessor, a computer, or multiple processors or computers. Theapparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Computer readable media suitable for storingcomputer program instructions and data include all forms of nonvolatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, beconsidered exemplary only, where exemplary means an example. As usedherein, the use of “or” is intended to include “and/or”, unless thecontext clearly indicates otherwise.

While this patent document contains many specifics, these should not beconstrued as limitations on the scope of any invention or of what may beclaimed, but rather as descriptions of features that may be specific toparticular embodiments of particular inventions. Certain features thatare described in this patent document in the context of separateembodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to a subcombination or variation of a sub combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

The invention claimed is:
 1. A method of processing video data,comprising: determining, for a conversion between a current video unitof a video and a bitstream of the video, that applicability of a firstcoding mode and a second coding mode is mutually exclusive, wherein, foran encoding operation, the first coding mode converts visual signalsfrom a first color domain to a second color domain, or for a decodingoperation, the first coding mode converts the visual signals from thesecond color domain to the first color domain; and performing theconversion based on the determining, wherein the second coding mode isan intra subpartition mode in which the current video unit is split intomultiple sub-regions that share a same intra mode, wherein in case thatthe first coding mode is enabled for the current video unit, the secondcoding mode is disabled for the current video unit.
 2. The method ofclaim 1, wherein when the second coding mode is enabled for the currentvideo unit, the first coding mode is disabled for the current videounit.
 3. The method of claim 1, wherein when the first coding mode isenabled, a third coding mode is enabled for a luma component of thecurrent video unit and disabled for a chroma component of the currentvideo unit.
 4. The method of claim 3, wherein in the third coding mode,differences between quantized residuals derived with an intra predictionand predictors of the quantized residuals are presented in thebitstream.
 5. The method of claim 3, wherein in the third coding mode,prediction samples are derived by performing a boundary down-samplingoperation on reference samples based on a size of the current videounit, followed by a matrix vector multiplication operation, andselectively followed by an up-sampling operation.
 6. The method of claim1, wherein when the first coding mode is enabled, a fourth coding modeis enabled both for a luma component and a chroma component of thecurrent video unit.
 7. The method of claim 6, wherein the fourth codingmode uses a piecewise linear mode to map a luma component betweendifferent domains, and a scaling coefficient derived based onreconstructed luma samples to scale a chroma residue.
 8. The method ofclaim 6, wherein the fourth coding mode splits a transform region intomultiple transform sub-regions and only one transform sub-region has anon-zero residual sample, and wherein a size of the transform sub-regionhaving non-zero residual sample is smaller than or equal to the othertransform sub-region.
 9. The method of claim 1, wherein the conversionincludes encoding the current video unit into the bitstream.
 10. Themethod of claim 1, wherein the conversion includes decoding the currentvideo unit from the bitstream.
 11. An apparatus for processing videodata comprising a processor and a non-transitory memory withinstructions thereon, wherein the instructions upon execution by theprocessor, cause the processor to: determining, for a conversion betweena current video unit of a video and a bitstream of the video, thatapplicability of a first coding mode and a second coding mode ismutually exclusive, wherein, for an encoding operation, the first codingmode converts visual signals from a first color domain to a second colordomain, or for a decoding operation, the first coding mode converts thevisual signals from the second color domain to the first color domain;and performing the conversion based on the determining, wherein thesecond coding mode is an intra subpartition mode in which the currentvideo unit is split into multiple sub-regions that share a same intramode, wherein in case that the first coding mode is enabled for thecurrent video unit, the second coding mode is disabled for the currentvideo unit.
 12. The apparatus of claim 11, wherein when the secondcoding mode is enabled for the current video unit, the first coding modeis disabled for the current video unit.
 13. The apparatus of claim 11,wherein when the first coding mode is enabled, a third coding mode isenabled for a luma component of the current video unit and disabled fora chroma component of the current video unit, and wherein in the thirdcoding mode, differences between quantized residuals derived with anintra prediction and predictors of the quantized residuals are presentedin the bitstream, or in the third coding mode, prediction samples arederived by performing a boundary down-sampling operation on referencesamples based on a size of the current video unit, followed by a matrixvector multiplication operation, and selectively followed by anup-sampling operation.
 14. The apparatus of claim 11, wherein when thefirst coding mode is enabled, a fourth coding mode is enabled both for aluma component and a chroma component of the current video unit, andwherein the fourth coding mode uses a piecewise linear mode to map aluma component between different domains, and a scaling coefficientderived based on reconstructed luma samples to scale a chroma residue,or the fourth coding mode splits a transform region into multipletransform sub-regions and only one transform sub-region has a non-zeroresidual sample, and wherein a size of the transform sub-region havingnon-zero residual sample is smaller than or equal to the other transformsub-region.
 15. A non-transitory computer-readable storage mediumstoring instructions that cause a processor to: determining, for aconversion between a current video unit of a video and a bitstream ofthe video, that applicability of a first coding mode and a second codingmode is mutually exclusive, wherein, for an encoding operation, thefirst coding mode converts visual signals from a first color domain to asecond color domain, or for a decoding operation, the first coding modeconverts the visual signals from the second color domain to the firstcolor domain; and performing the conversion based on the determining,wherein the second coding mode is an intra subpartition mode in whichthe current video unit is split into multiple sub-regions that share asame intra mode, wherein in case that the first coding mode is enabledfor the current video unit, the second coding mode is disabled for thecurrent video unit.
 16. The non-transitory computer-readable storagemedium of claim 15, wherein when the second coding mode is enabled forthe current video unit, the first coding mode is disabled for thecurrent video unit.
 17. The non-transitory computer-readable storagemedium of claim 15, wherein when the first coding mode is enabled, athird coding mode is enabled for a luma component of the current videounit and disabled for a chroma component of the current video unit, andwherein in the third coding mode, differences between quantizedresiduals derived with an intra prediction and predictors of thequantized residuals are presented in the bitstream, or in the thirdcoding mode, prediction samples are derived by performing a boundarydown-sampling operation on reference samples based on a size of thecurrent video unit, followed by a matrix vector multiplicationoperation, and selectively followed by an up-sampling operation.
 18. Anon-transitory computer-readable recording medium storing a bitstream ofa video which is generated by a method performed by a video processingapparatus, wherein the method comprises: determining, for a currentvideo unit of the video, that applicability of a first coding mode and asecond coding mode is mutually exclusive, wherein, for an encodingoperation, the first coding mode converts visual signals from a firstcolor domain to a second color domain, or for a decoding operation, thefirst coding mode converts the visual signals from the second colordomain to the first color domain; and generating the bitstream based onthe determining, wherein the second coding mode is an intra subpartitionmode in which the current video unit is split into multiple sub-regionsthat share a same intra mode, wherein in case that the first coding modeis enabled for the current video unit, the second coding mode isdisabled for the current video unit.
 19. The non-transitorycomputer-readable recording medium of claim 18, wherein when the secondcoding mode is enabled for the current video unit, the first coding modeis disabled for the current video unit.
 20. The non-transitorycomputer-readable recording medium of claim 18, wherein when the firstcoding mode is enabled, a third coding mode is enabled for a lumacomponent of the current video unit and disabled for a chroma componentof the current video unit, and wherein in the third coding mode,differences between quantized residuals derived with an intra predictionand predictors of the quantized residuals are presented in thebitstream, or in the third coding mode, prediction samples are derivedby performing a boundary down-sampling operation on reference samplesbased on a size of the current video unit, followed by a matrix vectormultiplication operation, and selectively followed by an up-samplingoperation.